Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process a challenging task due difficulty of choosing proper rank. In order address this challenge, paper proposes low-rank Bayesian tensorized network. Our method performs automatic model via adaptive rank determination. We also present approaches for posterior density calculation maximum posteriori (MAP) estimation end-to-end our provide experimental validation two-layer fully connected network, 6-layer CNN 110-layer residual network where work produces 7.4 × 137 more compact from while achieving high prediction accuracy.